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61 .  Introduction . 

In  the  preceding  papers  of  this  series  [l] ,  ,  we  have  studied 

two  particular  classes  of  f  motional  equations  which  arise  in  the 

theory  of  dynamic  programming.  The  first  paper,  [l  presented 

some  results  concerning  the  equation 

(1)  f(P)  =  Max  [R(P,Q)+h(P,Q)f(T(P,Q))J, 

Q 

(a  detailed  exposition  is  to  be  found  in  [2j ) ,  while  the  second  paper 
of  the  series,  [3J  ,  was  devoted  to  a  discussion  of  equations  of  the 
form 

(£,  dx^  »  Max  f1(xl,xa,. ..,xn,t;Q), x1(0)-c1, 1*1,2, ...n. 

dt  Q 

Equations  of  the  first  type  arise  from  the  study  of  discrete 
processes,  while  equations  of  the  second  type  arise  from  the  study 
of  a  certain  class  of  continuous  decision  processes.  Partial  differen¬ 
tial  analogues  of  (?)  arise  from  the  calculus  of  variations,  and 
from  the  theory  of  integral  equations,  cf  w .  w . 

All  of  these  equations  pertain  to  one— person  decision  processes. 

In  this  paper,  we  shall  study  a  class  of  functional  equations  arising 
from  the  theory  of  multi-stage  games.  The  equation  we  shall  use  to 
illustrate  our  techniques  is 

(?)  f(P,P')  -  Max  Min  [«/’/’  [R(u,v )  +h(u,  v)f  (T.T'  )JdG(u)dG'  (v)J 

G  G'  D(P,P») 

=  Min  Max  [  ...  J , 

G’  G 

where  T  -  T(P,P';u,v),  T' -T ' (P , P ’ ; u , v  ) ,  and  G(u),G'(v)  are  distribution 
functions  for  u  and  v  respectively  over  the  allowable  regions.  To  slmpl— 
fy  the  notation,  we  wr^te  R(u,v)  and  h(u,v),  although  we  actually  allow 
these  functions  to  depend  upon  P  and  P'. 


(2) 
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The  precise  restrictions  we  shall  impose  upon  the  functic  is 

which  appear  will  be  discussed  below. 

The  earliest  formulation  of  multi-stage  games  in  terms  of 

functional  equations,  of  which  we  are  aware,  is  contained  in 

R.  Bellman  and  J.  La  Salle,  [  11  J ,  and  R.  Bellman  and  D.  Blackwell, 

£  10  J  ,  where  "games  of  survival"  are  introduced.  Following  this, 

further  studies  of  the  existence  and  uniqueness  of  the  solution, 

M. 

together  with  properties  of  the  solution,  are  contained  ln/Pelsakoff 
£  13  J #  and  oellman,  £  6  J .  The  first  paper  stating  some  general 

existence  and  uniqueness  theorems  for  other  classes  of  multi-stage 

L. 

games  is  that  of/Sbapley,  [15  1  •  Since  then,  a  number  of  papers 
have  appeared  on  the  subject  of  multi-stage  games.  The  subject  has 
attracted  a  great  deal  of  attention,  and  deBervealy  so,  since  toe 
theory  of  multl-3tage  games  constitutes  a  natural  extension  of  the 
Von  Neumann-Morgenstern  theory.  In  some  sens?*,  we  may  even  consider 
the  multi-stage  process  as  basic,  giving  rise  to  the  single-stage 
theory  as  a  limiting  case  corresponding  to  a  "steady  state".  This 
is  a  clear  Inference  from  the  E. own-Von  Neumann  Iterative  solution 
of  games .  A  concept  of  this  type  is  useful  In  discussing  trie  play 
of  n — person  games  and  non — zero  sum  games.  For  an  application  of 
this  idea,  see  Qo  J,  wnere  the  Idea  Is  applied  in  a  neurlatlc  fashion. 

We  shall  begin  our  discussion  in  the  following  section  wltn  Lne 
description  of  a  mulul— stage  game  arising  from  the  study  of  two-person 
allocation  {recesses .  The  "principle  of  optimality",  [7j ,  will  be  used 
to  show  that  we  may  reduce  the  study  of  the  N-stage  process  to  the 
study  of  a  certain  system  of  recurrence  relations.  We  snail  then  con— 
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alder,  in  turn,  the  corresponding  process  Involving  an  unbounded 
number  of  stages,  the  process  where  the  Interaction  Is  stochastic 
rather  than  deterministic,  and  finally,  seme  time— dependent  cases. 

In  thl3  way  we  shall  be  1 ed  to  consider  the  equation  In  ()). 

Using  this  equation  as  cur  model,  we  shall  turn  to  a  discussion 
of  existence  and  uniqueness,  under  various  hypotheses  concerning 
the  coefficient  functions.  Our  proofs  will  depend  upon  the  method 
of  successive,  approximations ,  and  a  lemma  which  exploits  the 
quasi-llnear  aspect  of  the  functional  equation. 

We  shall  demonstrate^? fie  strategies  determined  by  the  functional 
equation  are  effective,  and  consider  the  stability  of  the  solution 
under  changes  in  R(u,v). 

The  method  we  shall  employ  in  presenting  the  results  sketched 

above  is  also  applicable  to  the  one-sided  functional  equation 

(4)  f ( P , P  ' )  -  Max  Min  [R(u,v)+h(u,v)f (T,T* )] , 

u  v 

as  was  pointed  out  to  us  by  W.  Fleming.  We  shall  also  use  a 
cruder  method  to  treat  this  problem. 

Finally,  we  shall  discuss  some  other  classes  of  multi-stage 
games,  such  as  "games  of  survival"  and  "pursuit  games",  which  lead 
tc  functional  equations  amenable  to  the  same  analysis. 

§2*  Descript  1  on  of  a  Multi-Stage  Gome 

•Let  us  now  describe  in  detail  the  multi-stage  game  we  wish  to 
analyze.  Two  player*- ,  whom  we  may  rather  prosaically  designate  by 
A  and  b,  possessing, respectively, resources  which  we  may  represent 
as  H-d  linens  lonal  vectors,  P  and  P',  are  engaged  in  a  multi-stage 
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process  carried  on  in  the  following  manner.  At  the  beginning  of 
each  stage  of  an  N— stage  process,  A  allocates  a  certain  quantity  of 
his  resources,  a  vector  u,  and  B  a  certain  quantity  of  his  resources, 
a  vector  v;  this  will  be  represented  symbolically  by  the  notation 
0<u<P ,  0<v<P 1 . 

As  a  result  of  this  allocation,  there  are  two  consequences.  A 
receives  a  pay-off  of  R(u,v),  a  scalar  function,  and  B  a  pay-off  of 
-R(u,v).  Furthermore,  their  resources  are  altered,  P  1b  transformed 
into  T(P,?';u,v)  and  P*  becomes  T* (P,P‘ ;u,v) .  The  process  now  continues 
in  the  same  fashion  for  ( M— 1 )  additional  stages. 

The  total  return  to  A  of  the  N-etage  process  Is  assumed  to  be 
additive, 

(1 )  Rn-Rn(u,U!  ,.  .  .  .u^jv.Vi  , .  . .  ,vN_T  )*R(u,v)-fR(ul  ,vt  )  +  ..  .♦Rfu^  .v^) , 

There  are  two  ways  of  treating  tne  N-stage  process.  We  can  either 
consider  the  N-stage  game  as  a  single-stage  game  of  complicated  type, 
requiring  a  choice  of  the  set  of  vectors  (u,ul , . . . ,uN_^ )  by  A,  and  the 

set  ( v ,Vj  , . . . , vM  . )  by  B,  where  the  choice  of  u  and  v.  depends  upon  the 

choice  of  u ,ui , . . . ,u^_j  ,v,Vi , . . • ,v^_j ,  or  we  can  use  the  functional 

equation  approacn  of  dynamic  programming,  [6J ,  [7j,  and  thus  reduce  the 
dimensions  of  the  process.  For  the  case  of  unbounded  processes,  or 
processes  Involving  st  )cnastlc  Interaction  of  u  and  v,  whlcn  we  shall 
discuss  below,  the  recurrence  relation  technique  seems  to  be  tne  only 
feasible  one,  while  in  the  case  of  finite  deterministic  processes,  this 
technique  Is  simpler  analytically,  conceptually,  and  computationally. 

Let  us  now  mane  some  assumptions  of  continuity.  We  snail  take 
R(u,v)  and  h(u,v)  to  be  a  continuous  function  :>f  u,  v,  P  and  P'  over 


(5) 
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■i 


any  finite  (P,P*;u,v)-  region,  and  similarly  for  ?(P,P’;u,v),  T'(P,P';u,?) 
to  be  continuous  functions  of  P,P',u  and  v.  The  case  where  P,P',u,v, 
R(u,v),T,T',  assume  only  finite  sets  of  values  la  also  Interesting  and 
may  be  treated  by  the  same  general  techniques. 

The  value  of  the  N-stage  game  described  above  is  given  by  the 
expression 

(2)  vN«rtax  Min  R^dG(u ,u, ....  ,u^_^  )dG '  (v, V| , . .  .  ,v^_1 )] 

•Min  Max  [  ...  J , 

G '  G 


where  G  and  G'  are  distribution  functions  over  regions  of  quite 
complicated  form  defined  by  the  Inequalities 

(3)  G<u<P ,  0<v<P’ , 

0<u,<T,  0<v i <T ' , 

°— UN-1-TN— 1  *  °<vn-i<t  ,n-i  . 

Note  that  T  and  T'  depend  upon  P,P’,  u  and  v,  Ti,T»'  depend  upon 
P,P' ,u,v ,uj ,vt ,  and  so  on. 

Obs<  rving  that  v^  depends  upon  P  and  P' ,  the  initial  states, 
and  only  upon  these  quantities,  let  us  define  the  sequence  of  functions, 
jfN( P,P')  I  ,  oy  means  of  tht  relation 
{*)  f N ( P , P • )  -  vN,  N-1,2,... 

53-  The  Pr Inc iple  of  Optimality. 

In  [7],  we  enunciated  a  principle  which  yields  the  functional 
equations  of  the  theory  of  dynamic  programming,  namely  the 
Principle  of  Optimality.  An  optimal  policy  has  the  property  that 

Jt 

i 

J 
i 

i 


(6) 


P-676 

5-15-55 


whatever  the  Initial  state  and  Initial  decisions  are ,  the  remaining 
decisions  must  constitute  an  optimal  policy  wltn  regard  to  the 
state  resulting  from  the  first  decisions . 

Applying  this  to  the  multi-stage  game,  we  obtain  the  following 
recurrence  relation 

(1)  fi(P,P')-Max  Min  </*  (P  R(u,v)dG(u)dG'  (v)-Mln  Max  [  ...  ] 

G  G '  0<u<?  G '  G 

0<v<P « 

fN.  1  (P.P')-Max  Min  t  (/  (R (u,  v  )+f n(T,T'  ) JdO(u)dG 1  (v)] 

"+1  GO'  0£u£F  " 

0<v^P ‘ 

'■Min  Max  £  .  .  .  J  . 

G*  G 

That  the  above  principle  Is  valid  fcr  one-person  processes  where 
we  are  attempting  to  maximize  a  return  or  minimize  cost  Is  clear 
by  contradiction.  Since  Its  validity  may  not  be  as  obvious  for  game 
processes,  let  us  present  a  brief  proof  for  the  Bake  of  completeness. 

The  recurrence  relation  in  (l)  provides  a  sequence,  not  neces¬ 
sarily  unique,  of  pairs  of  distribution  functions,  £ G^(u,P, P ' ) , G ' ^ ( v , P ,P ' 
which  furnish  the  sequence ^f^(P , P 1 )  <  .  In  order  to  show  that  the 
function  fN(P,P')  Is  actually  the  value  of  the  N-stage  game,  It  Is 
sufficient  to  show  that  A  can  guarantee  an  expected  return  of  f^jP.P') 

If  ho  chooses  u  at  the  first  stage  of  an  N-stage  process  in  accordance 
with  the  distribution  function  GN(u,P,P?),  when  the  states  of  A  and 
B  are  described  by  P  and  P',  respectively,  and  similarly  that  B 
can  guarantee  an  expected  loss  of  not  mere  than  — f N ( P , P * ) . 


(?) 


P-676 

5-13—55 


To  demonstrate  this,  consider  the  one— person  N— stage  process 

In  which  A  employs  the  fixed  strategy  represented  by  the  sequence 

> 

(u,P,P')|,  k-l,2,...,N,  and  B  attempts 
to  minimize  A’s  expected  N-stage  return.  It  la  sufficient  to  consider 
this  process,  since  any  other  policy  employed  by  B  yields  a  larger 
expected  return  for  *.  Let 

(2)  wN(P#P')  ■  N— stage  expected  return  to  A  when  A  employs 

the  fixed  strategy  £Gn(u,P,P' ,  B  employs 
a  minimizing  strategy,  and  A  and  B  are  in  the  initial 
states  P  and  P' . 

Then  we  have  the  recurrence  relations 

(3)  «t(P,P')  *  Inf  </•  Cl/5  F(u,v)dG(u,P,P')]dG'(v), 

G'  °<v<p‘  °<u:'p 

-  Inf  /  [  i/°  [r(u,v)+wn(T,T')Jc1G  (u,P,P')JdG»(v 

N*A  G'  o<v<P»  0£u<P  N  N*A 

upon  employing  the frlnclple  of  optimality  for  the  one— person  process. 

Considering  the  origin  of  the  function  F|,  we  see  that  the 

minimum  in  the  relation  for  wi(P,P*J  in  (})  is  attained  by  the  function 

G'-Gi',  not  uniquely  In  general.  Hence, 

(4)  w,(P,P')  -  v  ,  (P.  t*  • ) . 

Since  wj=vi,  the  relation  for  w2  yields  in  the  same  way  the  fact 
that  w 2*v2,  and  thus,  inductively,  we  see  that 

(b)  «N(P,P')  =  vN(P,P'). 

In  precisely  the  same  way  we  show  that  if  B  employs  the 
strategy  ^G'N (v ,P , P ' ) j,  A  cannot  cbtaln/^i^n  vN(P,P')«  Hence 
vN(P,P')  Is  the  value  of  the  N-stage  gfme. 

•  Related  Clashes  of  G^mes . 

Proceeding  formally  for  the  moment,  without  regard  to  the 
existence  of  the  quantity  we  define,  let  us  consider  the  unbounded 


of  distribution  functions 


■ 


(8) 
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process.  Define 

(1)  f(P,P')  ■  the  value  of  the  infinite  stage  process  to 

A  when  A  has  P  initially,  B  has  P',  and  both 
players  employ  optimal  strategies. 

If  f(P,P')  exists,  It  satisfies  the  equation 

(2)  f(P,P»)  -  Max  Min  [/  /  [h(u,v )+f (T,T ’  )]dG(u)dO*  (v)J 

G  G '  0£u<P 

0<v<P ' 

-  Min  Max  [  ] , 

O'  G 

provided  that  Max-Mln  -  Min-Max.  The  legitimacy  of  this  will  be 
discussed  in  the  following  sections,  under  suitable  assumptions. 

Let  us,  however,  observe  briefly  how  more  general  processes 
can  give  rise  to  various  extensions  of  (2).  If  we  allow  the  process 
to  be  time-dependent  In  the  sense  that  the  return  from  the  k— 
stage,  as  well  as  the  trans format  ions  T  and  T',  depends  upon  k, 
in  place  of  the  function  defined  by  (1),  we  must  consider  the  sequence 
of  functions 

(3)  f(P,n,;k)  »  the  value  to  A  of  the  Infinite  process  begin¬ 

ning  at  the  k**k  stage  when  A  possesses  P 
at  this  stage  and  B  possesses  P',  and  both 
employ  optimal  strategies. 

This  sequence  satisfies  the  recurrence  relation 

(4)  f(P,P';k)  -  Max  Min  [  /  /  [R(u,v,k)+f(Tk,TV;k+l)Jdg(u)dG'(v) 

G  O'  0  <u<P 

0  <v<P» 

■  Min  Max  [  ...  ] . 

G  G ' 

Let  us  now  complicate  the  process  to  a  further  degree.  We  have 
assumed  in  the  above  formulation  that  the  Interaction  between  the 
players  was  perfectly  determined  once  u  and  v  were  chosen.  It  is 
interesting  occasionally  to  consider  more  general  processes  in  which 
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a  choice  of  u  and  v  merely  determine  a  distribution  of  outcomes, 

denoted  by  the  function  (z ,  t ,  t 1 ; u, v )  ,  which  depends  upon  the  stage, 

is 

where  z/the  value  of  R(u,v),  t  the  value  of  T,  and  t'  the  value  of 


T'  . 

(5) 


Then  (4)  is  replaced  by 


f  (P , P ’ ; k )  -*ax  Min  [’(/'(/’ 
u  0 '  0<u<P 

O^v^P • 


fy0  [z+f  (t,t  ’  ;k+l)]dKk(z,t,t  '  ;u,v)] 

dO(u)  dO»(v)j 


-Min  Max  [  ...  3 • 

G '  G 

Finally,  let  us  consider  the  case  where  we  are  not  interested 
in  the  sum  of  toe  returns,  but  in  some  nonlinear  function  of  the  total 
return.  A  particularly  Important  example  is  the  probability  of 
achieving  a  return  of  at  least  Rq .  This  is  the  expected  value  of 
the  function  defined  by 

(6)  4  (u)  •=  0#  °£U<R0> 

3  l>  u>V 

aR 

Another  Interesting  utility  function  is  e 
describe  the  general  non-linear 

To  /  situation,  we  must  introduce  an  additional  state 

variable,  a,  the  return  obtained  by  A  from  the  previous  stages 

of  the  process.  Defining  f(P,P’,a;k)  essentially  as  in  (3)»  w* 


obtain  the  functional  equation 

(7)  f  ( P , P  1  a ; k ) *•  Max  Min  [Z  Z  l  Z  f  ( t , t  • , a  vz; k  +  1 )dKk (z , t ,  t ' ; u, v )J 

G  G«  0<u<P 

0<v<P  *  dQ (u)dQ ' ( v ) 


*  Min  Max  [  •  •  •  J 

G '  G 


We  shall  net  consider  any  of  these  more  complicated  functional 
equations  since  the  basic  aporoach  is  the  same  'n  all  cases,  despite 
the  fact  that  the  analytic  details  increase  in  complexity. 


(1C) 
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A  genuinely  new  class  of  functional  equations  emerges  from 
the  study  of  "learning  processes",  f 1 4 J  #  [9].  Here  It  is 
assumed  that  the  distribution  function  K(z,t,t ' ;u,v)  exists,  but 
Is  not  completely  known.  In  the  course  of  carrying  out  the  process, 
additional  Information  is  obtained  concerning  K.  The  problem 
la  once  again  to  maximize  the  expected  value  of  the  total  return. 

It  is  clear,  of  course,  that  we  are  encroaching  upon  the  domain  of 
sequential  analysis.  We  shall  consider  the  functional  equations 
obtained  in  this  way  in  a  subsequent  work. 

The  methods  we  employ  here,  combined  v>lth  those  uBed  to 
establish  the  results  of  [3] ,  can  be  used  in  very  much  the  same  form 
to  treat  the  nonlinear  differential  equations 
(11)  dx. 

-  •  Max  Min  [  ^ f,  (x,  t ;u , v )dG(u  )dQ '  (v)] 

dt  GO1  1 

-  Min  Max  [  ...  J,  1-1,2,... ,n, 

O'  G 

and  similar  types  of  lntegro-dlfferent lal  equations,  cf.  [l6J  ,  [  1 7 J  . 
65*  Statement  of  Principal  Results . 

Before  stating  our  results,  let  us  Introduce  some  notation 
and  definitions.  We  shall  take  P  and  P'  to  be  n-  and  n * -dimensional 
vectors  defined  over'  regions  D  and  D'  respectively,  each  contain¬ 
ing  the  origin  In  its  respective  space.  For  all  values  of  u, 
v,  P  and  P',  the  transformed  vectors  T(P,P';u,v),  T' (P,P' ;u,v) , 

are  required  to  lie  within  these  same  domains,  where  u  and  v  are 
k  and  k '-dimensional  choice  vectors  respectively,  constrained  to 
domains  S  and  S'  which,  in  general,  depend  upon  P  and  P'.  Since 
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we  are  dealing  with  shrinking  t rmns forma 1 1 ona ,  there  is  no  loss  In 
-assuming  D  and  D'  to  be  finite. 

In  each  space,  let  us  Introduce  the  norm,  ||P||,  equal  to  the 
sum  of  the  absolute  values  of  the  components  of  P, 

(’)  l|p||  -  E  |p.|. 

1-1 

llP'll  -  E  |P’,|. 

1-1  1 


Actually,  these  need  not  be  toe  same  norms,  and  in  some  situations, 
it  might  be  useful  to  consider  norms  molded  to  the  structure  of  the 
functional  equations  arising,  rather  than  standard  norms  of  the 
above  type. 

The  functional  equation  we  shall  consider  Is 


(2) 


whe  re 


f(P.P') 


Max  Min 
G  G' 


[  S  S  [ii(p.p';u.») 

vcl[s  '  (p!p*  )  +h(P.P'.u.v)f(T,T,)]da(u)d<?fc 


Min  Max  [  ...  J , 
O'  G 


(3;  T  -  T(P,P'ju.v), 

T’  -  T'(P,P'iu,v) 

To  simplify  our  notation,  let  us  represent  the  operator  appearing 
within  brackets  In  equation  (2)  by  T( P , P ' ; f ;G,G ' ) ,  so  that  the  above 
functional  equations  take  the  form 

(4)  f(P,P«)  -  Max  Min  T(P,P' ;  f  ;G,G' ) 

G  O' 

-  Min  Max  T(P,P' ; f ;G,G* ) . 

O'  G 


(12) 
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Whether  one  wishes  to  call  this  one  functional  equation  or 
a  pair  of  linked  functional  equations  appears  to  be  a  matter  of 
taste . 

The  result  we  wish  to  prove  Is 

Theorem  1.  Consider  tbe  above  equation ,  (4),  under  the  follow¬ 
ing  assumptions. 


(5)  (a)  The  functions  RCPyP^u^v),  h(P,P';u,v),  T(P,P';u,v) 

and  T ( P , P * ; u , v )  are  continuous  functions  of  P  and  P 1 , 
u  and  v ,  In  any  bounded  regions  of  tne  variables . 
f b )  The  choice  domains ,  S  ( P ,  P '  ) ,  S '  ( P ,  P  * ) »  vary  contin¬ 
uously  with  p  and  P' . 

(c)  T  and  T'  are  shrinking  t rans formations ,  l.e. 

Max  (||  T(P,P';u,v)|  |  +  |  |  T‘,r,P'  ;u,v)  |  | )  <  k  ( I  I  P|  |+ 1 1  p'|  I ) 

u«S 

v«S ' 


where  k  Is  a  fixed  constant  less  than  1. 


(d)  Let 

w(c) 


Then 


Max 

I  I P| I  +  I l P 1  I  I <c 


w(knc)  <  oo . 


(Max  |R(u ,  v  )  |  ) 
u€£  , 
v£S ' 


(e)  Max  |  |  h{P,P' ;u ,v) |  |  <1. 

u,v,P,P* 

If  the  above  conditions  are  satisfied ,  we  can  assert  tfiat  there 
Is  a  unique  solution  of  (4)  within  the  class  of  func tlons  f(P,P*) 

*For  simplicity,  let  us  suppress  the  P  and  P'  In  R(P,P';u,v). 


(13) 
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which  are  continuous  for  all  finite  P  and  P'  and  vanish  when  P' 
and  P'  are  both  null  vectors. 

This  solution  may  be  found  by  the  method  of  successive  approx- 
imations , 


(6) 

f  (P,P*)  -  Max  Min 

l  ^  $  R(u,v)do(u)da<(v)  ]  , 

G  G ' 

ueS , 

v«S  ’ 

-  Min  Max 

[•••], 

G'  0 


fn.1(P,P')  -  Max  Mln  T(P,P'jf  :0,G'), 

n  1  GO' 


-  Min  Max  T(?,P'  ;f  ;G,Q'  ) ,  n  >  o, 
0 '  G 


with  f  ( P ,  P '  )  -  11m  fn(P#P')  iri  any  bounded  domain  of  the 


( P , P ' )  space  . 


We  shall  further  demonstr  ;e 

Theorem  2 .  Under  the  hypotheses  of  Theorem  1 ,  a  set  of  functions , 
(Gvu) #Q* (v) )  ,  furnished  by  the  functional  equation  constitute  a  set 
of  optimal  strategies  for  A  and  B  respectively  In  the  multi-stage 
game  described  In  the  pi-ecedlng  sections. 


(14) 
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56 .  Lemma . 

Let  us  present  a  simple  but  extremely  useful  Inequality  which 


exhibits 

the  quasl-llnearlty 

of  the  transformation 

0) 

L(f)  -  Max  Min 

T(P , P ' ; 

0,  G1 ) 

■  Min  Max  T 

G  O’ 

G*  G 

Lemma  1? 

Let 

(2) 

L(f)  -  Max  Min 

/ 

R(u,v)  + 

h(P,P' ;u , v ) 

f (T,T '  ) 

G  G« 

-  u«S 

- 

- 

veS* 

dG(u)dG' 

-  Min 

Max 

r  ... 

1, 

G* 

0 

L 

J 

Li (P)  -  Max 

Min 

'  / 

f  [Ri(u,v)  +  h( P, P  * ;u  ,  v )F(t,T ' ) 

G 

O' 

-  ueS 

v*S‘ 

dG(u )dG ' 

-  Min 

Max  I 

•  • 

1* 

G» 

0 

- 

J 

Then 

O) 


|L(f)-Li (P)  |  <  Max  Max 

G  0» 


J1  (f  I  | B ( u , v ) -« , (u,v) |  + 
u«S  1 

veS' 


|h(P,P';u,v)|  Jr(T,T’ )-F(T ,T ’ ) I J  dg(u)dG'(v)  . 

Proof;  Let  us  write 

(4)  L(f)  -  Max  Min  T ( P , P ' ; f ;G  ,G ' )  -  Min  Ma x 

G  G'  G '  G 

Li(F)  -  Max  Min  Tj  (  P ,  P  '  ;F  ;  G  tG  • )  -  Min  Max 

u  G •  G'  G 

Let  (Oi.Oi ')  be  a  pair  of  functions  yielding  tne  value  L(f),  and  (ua  ,Ga  '  ) 

be  a  pair  of  functions  yleldln?;  the  value  L »  ( F ) .  Then,  by  virtue  of  the 

saddle-point  property, we  have  the  following  chain  of  equalities  and 
Inequalities _ 

It  Is  assumed  that  max-mln  •  mln-max  for  each  transformation.  A  similar 
result  holds  for  the  one-sided  max-mln  operator;  see  £  14. 
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(5)  1(f)  •  T(P,P';f;Oi,Gi')  >  T(P ,P ' ; f ;GS,G , ' ) 

<  T(P,P';f;0i,0*'), 

Li(F)  -  T,(P,P'iF;G*,G2')  >  T , (P,P • ;F; C, ,GS ' ) 

<  T  i  ( P ,  P  ' ;  Fj  Otp  i ' ) 

Combining  these  inequalities  we  have 

(6)  L(f)-L|(F)  >  T(P ,  P' ;  f ;  02f  G, 1 )  -  T ,  (P.  P 1  ;F;Ga,G , 1 ) 

<  T(P ,P 1 ; f ,G i ,Ga ' )  -  T,(P,P';F;Q,.C,') 

The  Inequality  In  (6)  yields 

(8)  L(f)-L,(F)  >  (/^/,^B(UiV)_Rl(u,v)+h(P,P';u,v)[f(T, TM¬ 


PC?, T')]j  dO,(u)dG,'(v) 

<  f  ,/  f  R(u,u)-R,(u,v)  + 
“  n^-n  L 


u<D 

vv-D 


h(P,P 1  i u, v)  [f(T,T')-F(T,T')jj  dG , (u)dG,1  (v ) . 

Using  the  fact  that  a  <  c  <  b  implies  |c  |  <  Max(|a|f  |b|),  we 
obtain  from  (8)  the  further  Inequality 

mtr 

(9)  | L(f )— L, (F )  |  <  Max  ,/’  />  U (u, »)-R , (u,' 

“  i,.  n  L 


v  ♦ 


v  I  UvD 
v<cD ' 


|h(F,P';u,v) j  | f (T,T' )-F(T,T' ) |  J  dG» (u)dC i ' (v ) 

I  c/}  (P  lR(u,v)-^j (u,v)  |  + 

L  u*D 


vtD' 


dG|(u)dG,'(v)  , 


|h(P,P';u,v) |  |f (T,T' )-F (T,T ' ) | 
from  which  (})  follows  Immediately. 

It  Is  easy  to  make  the  modifications  required  to  obtain  the  analogous 
result  for  the  case  where  Max  l^n  Is  replaced  by  Sup  Inf. 

$7 .  Exl stence  and  Uniqueness  . 


We  can  now  proceed  to  a  proof  of  Theorem  1. 
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Let 

( 


f  (P,P' )  -  Max  Min 

°  G  G' 


Min  Max 
G'  G 


(/'  (P  R(u,v)dG(u)4ff(v) 
L  ueS  j 

veS  1 


and 

(2) 


n*l (P, P 1 )  -  Max  Min  T(P,P’;f  ;  G ,  G  ’  )  -  Min  Max  T, 


G  G' 

where  T  Is  defined  as  In  (4.2)  and  (4.4). 


n' 


By  virtue  of  our  assumptions  concerning  the  coefficient  functions, 
and  the  domains,  S  and  S', 

/we  can  assert  the  existence  of  the  saddlepoint  in  (l),  and  the 
continuity  of  fo(P,P1)-  Inductively,  then,  all  the  f  [?,?')  exist 

and  are  continuous  for  all  finite  P  and  P'. 

( 

Let  us  now  show  that  the  sequence  j  f  \  converges  uniformly  in 
any  finite  portion  of  the  (P , P 1 )-reglons .  Using  Lemma  1  we  obtain 


the  Inequality 
(3) 


f  (P.P') I  <  Max  Max 

11+1  n  _  r.  r.  i 


G 

dG 


<J’  '/>  |l'n(T,T')-fn_1(T,T') 
(u)dQ’ (v ) J  ,  n-2, J . 


Define  the  new  sequence 


(*) 


u 


n+1(c)  -Max  lrn..(P,P')-rn(P,P')|. 

I  I P  I  I  +  I  I  P  '  I  I  <o  n+i  n 


Then  (3)  yields,  using  the  assumption  of  (4a)  of  ^3, 
(5)  un+i  (c  )  <  un(kc ) ,  n-2, 3,  .  .  .  , 


Alsq  we  have 
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(6)  |f»(F,P,;-rn(P,P')l  <  Max  Max  i/'  J1  |R(u,vJilG(u)dO'  (v) , 

“  0  G' 

whence 

(7)  u2(c)  <  w(c)  . 

Using  cur  assumption  that  Z  w(knc)  <  cc,  we  see  that  the  series 
Z  (P,P' )-fn(P#P'  )J  converges  uniformly  in  any  finite  region. 

Hence  f  (P,P‘)  converges  uniformly  to  a  function  f(P,P')  which 
satisfies  the  original  functional  equation. 

This  completes  the  proof  of  existence.  Let  us  now  turn  to  a 
proof  of  uniqueness.  Let  F(P,P')  be  another  solution  which  is 
continuous  at  P-0,P'»0,  and  bounded  in  any  finite  region.  We  see 
that  F( P , P ' )  is  then  actually  continuous  for  all  finite  P  and  P', 
although  this  fact  is  not  necessary  for  our  proof.  It  does  simplify 
it  a  bit  since  we  can  replace  Sup — Inf  by  Max-Mln. 

We  then  have  the  two  equations 

(8)  P(P,P‘)  -  Max  Min  T(P,P*  ;P;Q,Q’ ) 

G  O' 

f(P,P')  -  Max  Min  T(P, P ' ;  f ; G,G • ) . 

G  G’ 

Applying  Lemma  1,  we  see  that 

(9)  | F ( P ,  P  *  )— f  ( P ,  P  1 )  1  <  Max  Max  (/>  |P(T,T' )— f  (T,T* )  IdGdG' . 

"  G  G'  L  utS 
v^S' 

Let 

(10)  Me)  -  Max  { F ( P  , P  * )— f  ( P , P ' )  |  . 

I  IP  I  l+l  |P*  I  I*  c 

Then  (9)  yields  the  relation 

(11)  Me)  <  M^c) , 

UiB  ) 
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which,  upon  Iteration,  yields  A(c)  £  A(knc),  11-1,2,...,.  Since 
P  and  f  are  both  continuous  at  P  ■  0,  P'  ■  0,  and  have  -he  common 
value  0  there,  we  see  that  A(knc)  0  as  u  — ^  a&  Hence  A(c)  -  0 
and  P  m  f. 

This  completes  the  proof  of  Theorem  1 . 

Successive  Approximations  in  Qeneral 

The  sequence  of  approximations,  ^fn(P,P')^  ,  we  used  to 

construct  the  function  f(P,P')  was  precisely  that  obtained 
the  sequence  of  values  associated  with 
from/the  finite  n— stage  processess.  This  is  actually  not  the 

best  sequence  to  use  if  we  are  interested  only  in  the  infinite 

stage  process.  As  we  nave  pointed  out  elsewhere, [  4  J ,  £7J, 

approximation  in  "policy  space",  here  "strategy  space",  is  in 

many  ways  a  ;no’e  natural  and  more  Important  type  of  approximation. 

To  Justify  this  and  other  types  of  approximations  we  require 

Theorem  3.  Under  the  assumptions  of  Theorem  1,  the  sequence 

defined  by 

(1)  f  ..(P,P')  -  Max  Min  T(P,P* ;f  ;G,G' ) ,  n-0,1 , . . . 

n+A  0  G’  n 

-  Min  Max 
G '  G 

Converges  to  the  solution  of  (5.3)  for  any  initial  function  f  ( P , P • ) 
which  is  continuous  in  any  finite  part  of  the  (P,P*  )-domaln, 
and  equal  to  zero  at  P-0,P'-0. 

The  proof  is  precisely  the  same  as  that  given  above. 

§  9*  Effectiveness  of  Solution . 

We  have  established  existence  and  uniqueness  of  the  functional 

the 

equation  we  derived  under  the  assumption  that/lnf lnite  process 
possessed  a  value  for  each  player.  The  question  now  arises  as 
to  whether  the  functional  equation  actually  yields  sufficient 
information  to  allow  each  player  to  obtain  this  value.  If  so, 


/  O  A\ 
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we  say  that  the  solution  Is  effective. 

To  show  effectiveness,  under  the  hypotheses  of  Theorem  1, 
we  must  show  that  If  A  uses  a  distribution  function  0(u)  •  0(u;P,P») 
obtained  from  a  pair  (0,0')  which  yield  the  rain-max,  then,  regard¬ 
less  of  what  B  may  do,  we  can  guarantee  a  return  of  at  least 
f(P,P' )  to  A. 


Employing  this  fixed  strategy,  A’s  return  will  be,  at 
worst,  determined  by  the  solution  of  the  functional  equation 


(1) 


F( P  , P • )  -  Min 

O' 


/  ^  [R(u,v)+h(P,P';u,v)P(T,T')]dia(u)dO'(v) 


v*  S 


It  Is  easy  to  show,  using  the  techniques  of  the  preceding  sections, 
cf  [2] ,  where  these  equations  are  treated  In  detail,  that  this 
equation  h*s  a  unique  continuous  solution  which  is  zero  at 
P-0,P'-0.  Furthermore,  the  solution  of  this  equation  may  be 
obtained  as  the  limit  of  the  sequence  defined  by 

(2)  ?(P,P’)  -  Min  [/  /R(u,v)dO(u)dG'(v)  1  , 

G'  |  u(S  J 

v«S' 

w--*  •  /I"'-  )+h(P,P» ;u,v)Fn(T,T' )dG(u)dO' (v)j 

v«S ' 

It  Is  clear,  from  the  derivation  of  G(u),  that  F  2  f» .  Hence, 
Inductively,  Fn+1  -  f  as  defined  by  (7.2).  Thus 

(3)  P(P.P')  -  Urn  P  -  11m  f  -  f(P,P'). 

n— >00  n — >ao 

This  demonstrates  the  effectiveness  of  the  solution  In  the 
continuous  case. 

£10.  Stability  of  the  Solution 

An  Important  aspect  of  any  physical  process  is  the  dependence 
of  measuring  functions  (In  this  case,  the  value  of  the  game), 
upon  the  parameters  and  coefficient  functions  which  determine 


(20) 
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the  process.  In  general,  we  expect  small  changes  in  these 
quantities  to  result  in  small  changes  in  the  measuring  functions. 

If  this  le  true,  with  appropriate  definitions  of  "smallness**, 
we  say  the  process  is  stablt .  A  principle  of  wide  validity  is  that 
physical  processes  art  stable.  Of  course,  since  the  mathematical 
transcription  of  any  physical  process  Js  never  precise,  we 
cannot  conclude  immediately  that  the  mathematical  process,  as 
defined  by  the  equation,  is  stable.  Actually,  one  of  the  most 
useful  tests  of  the  realism  of  a  mathematical  model  of  a  physical 
process  is  that  of  stability. 

Let  us  now  establish 
Theorem  4.  Let 

(1)  A(c)  -  Max  Max  | R(u,v)-R ’ (u ,v) | . 

I  I  p  I  I  + 1  I  p  ’  I  I  <c  u«S 

v«S' 

Then,  under  the  hypotheses  of  Theorem  1 ,  the  solutions  of 


(2) 


f(P,P')  -  Max  Min  [/  /  [R(u,v)+h(P,P'  ;u,v)f(T,T' )|dGdO 

G  G'  r  u*S  L  J 

v*S 


Min  Max  F 
G'  G  L 


]. 


P(?,P') 


Max  Min 
G  G' 


^  tf/’R,(u,v)+h(P,P'  ;u,v)F(T.T'  jJdGdG'J 


ucS 

y*S' 


«  Min  Max 
G*  G 


(21) 
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satisfy  the  Inequality 

(3)  |f(P,P,)-F(P,P')l  <  ®  A(kno). 

n-0 


Proof : 

(4) 


Applying  the  Lemma  of  §3#  we  see  that 


f  ( P ,  P '  )—F  ( P ,  P 1 )  I  < 


Max  Max 

0 


O'  [<£  { [' 


v«  S' 


R— H ’  |+|f (T,T'  )— F (T* T ' ) | 
dGdG  J  . 


Iteration  of  this  Inequality  yields  the  desired  result. 


611 .  Further  Results . 

There  are  a  number  of  different  ways  In  which  the  results  of 
Theorems  1  through  4  can  be  extended  and  generalized. 

These  results  depended  upon  the  fact  that  the  transformation 
(T,T')  was  a  shrinking  transformation ,  In  the  sense  explained  above. 
An  Intuitive  visualization  of  this  Is  to  consider  (P,P')  as  repre¬ 
senting  the  resources  of  each  side.  Then  each  play  of  the  game 
diminishes  the  total  resources  available. 

We  can  Introduce  a  shrinking  trims  formation  In  another  way 
by  Imposing  the  conoltlon  that 


0) 


|h(P,P' ;u ,  v ) |  <  k  <  1 . 


A  condition  of  this  type  can  »*rlse  In  two  ways.  First  of  all, 
it  may  represent  the  discounted  value  of  future  actions  as  contrasted 
with  the  present;  secondly,  It  may  represent  a  probability  of 
survival  In  situations  In  which  there  is  a  non-zerc  probability  of 
the  termination  of  the  process  associated  with  every  play  of  the 

(22) 
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game  of  £16.  Both  concepts  are  connected  through  the  intermediary 
of  prediction  theory. 

Having  Introduced  (1),  it  Is  no  longer  neceasary  to  assume 
that  (T,T*)  Is  a  shrinking  transformation.  We  must,  however, 
assume  that  (P,P‘)  nd  transforms  always  xle  within  some  fixed 
region.  With  these  conditions,  the  analogues  of  Theorems  1 
through  4  are  readily  obtained. 

Ihirthermore ,  similar  results  may  be  obtained,  using  the 
same  methods,  for  the  generalized  equations  mentioned  In  §4, 

under  various  combinations  of  the  above  assumptions. 

Equations  satisfying  either  of  the  above  conditions  correspond 

to  the  equations  of  Types  One  and  Two  discussed  In  our  paper  on 
one-person  processes,  [2].  When  we  consider  other  types  of  processes, 
the  analysis  becomes  more  specialized  and  complicated,  cf.  [2] ,  $10. 

In  a  different  direction,  we  mey  ~elax  the  restrictions  of 
continuity  which  we  have  Imposed  and  investigate  the  conditions 
under  which  we  obtain  solutions  to  equations  of  the  form 

f(P,?')  -  Sup  Inf  T(P,P';f;a,G’), 

0  O' 


-  Inf  Sup  T( P , P ’ ; f ; 0 , 0 • ) . 

O'  o 

In  these  cases,  we  will  obtain  Active  strategies. 

9  2.  Dlffe rent lability. 

If  we  assume  that  under  suitable  assumptions  of  concavity  and 
convexity  the  functional  equation  reduces  to 

(1)  f ( P, P * )  -  max  Min  I  r(u , v )+f (T ,T ' ) 

u  v  j 

then,  In  certain  other  fortunate  cases,  we  can  reduce  the  equation 

(*3) 
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to  a  functional  equation  of  more  conventional  form. 

To  Illustrate  the  Idea,  consider  the  functional  equation 

(2)  1 -!x)  -  Max  f  g(y )+h (x-y )+f  (ay+b (x— y ) )  1  , 

°<y<x J 


where  0<a,b  <1,  which  we  have  discussed  in  a  number  of  papers,  2]  7 
If  the  raaximum^^&^Ps  in  the  interior  of  the  interval,  end  if  we 
assume  that  g  and  h  are  differentiable,  we  obtain  the  two  equations 
(2)  g' (y)-h' ( x-y ) 4^— b ) f ' (ay+b (x-y )  )-0 

f ' (x)-h' (x-y)+bf  * (ay+b (x— y ) ) , 

which  allow  us  to  compute  f'(x)  via  a  relatively  simple  recurrence 
relation . 

Similarly,  if  we  consider  the  equation 

(“)  “  Mm  'lln  rR(u,v;x,y)+f(T(x,y,u,v),T'(x,y,u,v) 

0£u <x  O^v^y  L 


-Nin 

0^u<x 


C  ••  } 


where  x  and  y  are  now  scalars,  and  assume  that  the  saddlepolnt 
always 

exists  and  ls/lnslde  the  region  for  all  norr-negatl ve  x  and  y, 
we  can  reduce  (4)  to  the  set  of  simultaneous  equations 


(5) 


VTufx(T,T')*T'ufy(T,T')"0’ 
VVx(T’T'  }+Vfy(T>T’ 


fx-VTxfx<T’T,)' 

f  -R  +T  f  (T.T" )  • 

y  y  y  y 

One— Sided  Min- -Max 


Let  us  now  consider  the  equation 

f ( P , P  * )«Mln  Max  j~  R(u,v)+h(P,P';u,v)f(T,T' 
v>  S'  u‘  S'L 


which  arises  frjra  the  allocation  process  described  above  if  the 
second  player  is  required  to  announce  his  choice  of  v  before  each 
play.  ^  ) 
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TTsing  a  technique  due  to  Wendell  Fleming,  we  can  treat  this 

equation  In  exactly  the  same  fashion  as  these  appearing  In  the 

previous  sections.  We  begin  by  noting  that  for  any  function  r(u,v) 

(2)  Mir  Max  R(u,v)  ■  Min  Max  R(u,v), 

v€*5'  USS  vfcS'  U(v)€S 

where  u(v)  Is  now  a  function  of  v  which  maximizes  R(u,vy  for 
fixed  v.  Let  u(v)  be  this  function. 

Let  V  be  a  value  of  v  which  minimizes  r(u(v),v).  Then  we 
have  the  two  Inequalities 
(4)  r(u(v) ,v)  -  <  r(u(v),v), 

R(u(v),v)  >  R(u(V),V), 

for  any  other  admissible  values  of  u  and  v. 

Using  these  Inequalities  we  readily  obtain  the  analogue 
of  the  lemma  given  in  §4  for  equations  of  the  above  type.  In 
this  way  we  can  establish  the  analogues  of  the  previous  theorems. 


An  Alternative 


Let  us  now  show  that  there  are  alternative  approaches  which 
cen  dispense  with  our  lemma,  and  rely  instead  directly  upon  the 
shrinking  properties  of  the  transformation.  Let  us,  for 
simplicity,  consider  the  most  Important  case  where  R(u,v)  and 
h(P,P‘;u,v)  are  non— negative . 


Let,  as  above 


(1)  w ( c )  -  Max  Mav  Max  R(u,v). 

|jP|  M  |  P  •  |  |<c  u*S  v<3’ 

As  usual,  let  us  introduce  the  sequence  ^fn(P,P‘)^ 

fi  (P,P»)-Mlr  Max  R(u,v) 

veS'  urS 

f  . (P,P ' )»Min  Max  |F(u,v)+h(P,P ' ;u,v)f  (T,T’ 
n+1  »«S'  u€S  L 


,  where 

)  » n»l , 2 ,  .  . 


(2) 
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Then 


fi(P,P')  <  f*(P,P')  <  Min  Max  f  R(u,v)+w(kc)  ] 

v^’S*  vi6S  -J 


<  f ,(P,P')+w(kc). 

Continuing  in  th-3  fashion,  we  see  that 


fa(P,P')  <  fs(P,P')  <  Min  Max  f R(u,v)+h(P,P' ;u,v)fa(T,T' )+ 

-  vO>  u«  t  w(a,c)j 

<  fe(P,P’)+w(k-lc). 
fn(P.P’)  S  fn+l|?-p,)  S  rn(?,T')+*^nc). 


Thus  we  have  uniform  convergence  to  a  solut'on,  under  our 

assumption  that  £P  w(knc)  <  oo  . 

n-1 

816 •  Probablll ty  of  Survival . 

briefly 

Let  us  now/ consider  some  other  classes  of  multi-stage 
games  which  lead  to  related  classes  of  functional  equations. 

To  begin  with,  let  us  consider  the  allocation  process  discussed 
In  §2  In  which  we  assume  that  there  Is  a  probability  h(P,P';u,v), 
dependent  upon  P,P',  u  and  v,  that  the  process  will  terminate 
at  the  end  of  the  partlcubr  stage.  The  functional  equation 
governing  the  process  is  then  the  equation  we  have  discussed 


in  the  preceding  sections  where  f(P,P’)  Is 


the  expected 


return  to  A.  In  the  fliocation  process,  h  may  be  either  1,  or 
a  "discount  factor" emphasizing  the  present  value  of  a  return  as 
opposed  to  a  future  value. 

§16.  Games  of  Survival . 

Associated  with  the  p.evlouB  concept  of  probability  of 
survival  Is  the  class  of  games  called  "games  of  survival".  Here 
both  players  are  actuated  by  the  desire  to  survive  the  other, 
with  each  play  of  the  game  involving  either  a  diminuatlon  of 
resources  of  one  or  both  players,  or  an  actual  chance  of  elimination. 


P-£7>' 


Here  the  basic  functional  equatl<n  Is 

(1)  f ( P , P 1 ) -Max  Hln  /  ,/)  h (P , P • ; u, v )f (T,T ' )dO(u )dG ' (v) 

G  O' 

v .  D ' 

-Min  Max  (  ... 

O'  G  l 

where  f  ( P # P * )  la  now  the  probability  that  A  survive  B.  The 
equation  la  valid  only  for  P , P ' >0  with  the  aide  conditions 

(2)  f(P.O)  -  1,  P>OJ 
f  (0, P 1 ) -  0 ,  P ' >0 , 

f(0,O)  -  1/?,  (as  a  matter  o f  convention). 


This  equation  la  very  much  more  difficult  to  treat  than  the 
foregoing  equations  In  the  cases  of  greatest  Interest  where 
h*l  rnd  T  and  T'  a-~  merely  restricted  to  lie  In  the  bounded 
regions  containing  P  and  P*. 

Particular  results  may  be  found  In  gj,  Mllror  and 

Shapley , jl2j .  This  approach  Is  occasionally  useful  In  treating 
non -zero  sum  games,  see  ]  yj. 

^17.  Games  of  Pursuit . 

Finally,  let  us  mention  the  very  Interesting  and  difficult 
problems  connected  with  pursuit  games.  There  is  ps  yet  no 
satisfactory  theory  of  continuous  pursuit  games,  which  necessarily 
restricts  us  to  a  discussion  of  discrete  games. 

Let  us  assume  that  two  players,  A  and  B,  are  restricted  to 
positions  at  the  lattice  points  of  the  plane.  A  can  move  up  to 
k  units  from  his  position  In  either  horizontal  or  vertical  direction, 
and  B  can  move  up  to  1  units  In  the  same  manner.  With  both  players 
required  to  move  simultaneously,  we  are  interested  In  the  strategies 
which  enable  A  to  catch  B,  to  catch  B  In  minimum  time,  or  to 

minimize  some  other  payoff  function. 

(27) 
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II  jlayei  >n  the  uiti<  .mied  region  of  the  plane,  It  le  not 
easy  to  determine  when  caj tur*  occurs,  and  the  j roblem  le  not 
trivial  for  »  bounded  region  either. 

Let  un  set 

(1)  l(P,P')  -  trie  time  required  for  A  to  catch  B  when 

A  la  at  trie  lattice  point  P,  B  le  at  the 
lattice  point  Q,  and  both  playere  employ 
optimal  strategies. 

Then,  without  Inquiring  Into  the  existence  of  our  function, 
the  equation  satisfied  by  f  Is 

(?)  (P,P')  -  It-  Min  Max  L/  /]  f  ( P+e ,  P  '  +  f  )dO(e  )dQ '  (f )  1  , 

G  O'  1. 

-  1+  Max  Min  [~ 

O'  G  L 

where  G(e)  and  G(f)  are  distributions  over  the  allowable 


vectors  e  and  f.  This  equation  holds  prior  to  capture.  At 

done 

capture,  the  process  terminates.  Very  little  has  beery^ln  connection 
with  establishing  existence  and  uniqueness  theorems  for  these 
equations.  We  shall  discuss  them  In  a  subsequent  paper. 

618.  Colonel  Blotl o . 

Let  us  now  consider  a  well-known  single-stage  game  which  can 
profitably  be  considered  to  be  a  multi-stage  process.  Let  A  and  B 
possess  trie  scalar  quantlt  es  x  and  y,  both  oosltlve. 

A  divides  x  ~nto  a  sum  of  N  non-negative  quantities  x » ,x* , . . . ,x^, 


and  B  dees  likewise  with  y,  bo  that  we  have 

(1)  x  «  x  i+x8+ . . *+xN,  x^>  0 

y  -  y i«-y •  -+yN- 

As  a  result  of  this  allocation,  A  receives  a  pay-off  of 

N 

(2)  RN^xl,yj^  '  ^  max  (*1-*!'°)’ 
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and  B  a  pay— off  of  —  R^x^y^). 

Let  us  define  fN(x,y)  to  be  the  value  of  this  game.  Then  we 
have  the  recurrence  relations 

x  y 

(3)  fi(x,y)  **  Max  Min  J'  Max  (x  i-y  i  ,0)dC  (^)dC' (j|) , 

G  O’  0  0 

x  y 

-  Min  Max  </*  J''  Max  (x  t-y  i  ,0)aG  (x^dG '  (a) , 

G'  G  0  0 

x  y 

fN. !  (x,y)-Max  Min  [  /  </"  [Max  (x  »-y  i  ,0)  +  f  N(x-x  i  ,y-y  i  )]dG(^)dG  '  ( a)] 
G  G'  0  0 


-Min  Max  £  ...  ~] 

G '  G 


This  formulation  facilitates  both  analytical  and  computational 
treatment . 


(29) 
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